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ABSTRACT 

The paper, one * of a series written as the Hanagement 
Information System for Occupational, Education (HISOE) was 
conceptualised, examines relationships between the anticipated census 
and sample data to be developed and maintained by the system. Two 
related information sources are planned: one is a census information 
system which includes data about every occupational education program 
in every school in every city and town in Massachusetts, and the, 
other is a sample information system by occupational programs, 
stratified over schooi-type, geographical setting -type f and student 
characteristic-type dimensions currently operative in the State. The 
types and purposes off the census and sample data to be collected and 
maintained are described. The descriptive data (input information, 
process information, product information, and impact information) and 
analytical data (cost accounting information, process product data, 
cost product information, product impact information, process impact 
data, and cost impact data) included in the sample data system are 
discussed at length, defining each type and showing how each data 
type is related to the census. (The author suggests that Monograph 
lumber 1, which is available as ED 062 553, be read in conjunction 
with this paper.) (Author/MS) 
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The purpose of this brief paper Is to examine relationships between 
the anticipated census and sample data to be developed and maintained by 
MISOE. None of the following Is conclusion oriented but rather 'conceived as 
our thinking at this early point In developmental time. This paper will also 
reconsider each data type of the system In light of the sample-population 
relationships. 

Essentially, MISOE provides a basis for describing what has occurred 
In occupational education In such a way that predictions for future outcomes 
can be optimized. Such Information should provide a management tool for Im- 
proving programs, allocating resources and maintaining an accountable re- 
lationship between state and local educational agencies. The very essence 
of the system Is to provide a state-wide data base for the management of oc- 
cupational education that Is stmultaneouslyuseful at both state and local 
levels, and at the same time one which meets all the information require- 
ments of the State Legislature, the Congress, etc* Two related Information 
sources are planned: one Is a census Information system which Includes data 
about every occupational education program In every school, In every city and 
town In Massachusetts, (Including those programs with proprietary Institu- 
tions), and the other Is a sample Information system by occupational programs, 
stratified over school-type, geographical setting-type, and'student- 
charactertstlc-type dimensions currently operative In the Commonwealth. From 
the state f s perspective, the Information system will provide a careful esti- 
mate of the totality of occupational education within the state. The census 
data will Identify specific local agencies but the sample data will not. 
The development of Instruments which will be used to measure elements of the 
educational process will be administered by the state within the selected 
sample and made available to local educational agencies. This provides a 



( 

basis for LEAs to compare themselves to the'appropriate strata within the 
sample and at the same time provides a basts for the state educational 
agency to conduct an educational audit at will. 

It Is Important to continue to think of the system under develop- 
ment as not only an Information system but a management system. The manage- 
ment and Information system Is designed to encourage (consistent' with the 
Monograph #1) appropriate flexibility for declston making at the local 
level , while at the same time provide a comparable state-wide data base. 
This Is difficult business. It Is Important that the system does not be- 
come a tool by which mindless bureaucrats can standardize educational prac- 
tices In the Commonwealth. 

I. CENSUS DATA 

The next section of this report presents a cursory examination of 
the census data that will be collected and maintained by the total Informa- 
tion system. 

The purpose of census data Is several fold: 

(1) To present an annual projection of enrollments, expendi- 
tures and specific performance objectives for management 
and accountability purposes, by program, by city or town, 
and by school . 

(2) To present an annual (historical) description of 
expenditures and enrollment in occupational education 
for management and accountability purposes by program, 
by city or town, and by school. 

(3) To establish a population base to draw a sample for 
more detailed analysis such that Inferences can be 
made to subsets within the population. 



Census data Is both a planning and accountability tool. It Is 
designed to provide the linkage between each LEA and the detailed analysts 
of the sample, while offering an Information base for Instructional manage- 
ment (See Figure I). 

A tricky census data requirement Is to provide expenditure Informa- 
tion for accounting and analysis, consistent with both needs. The expendi- 
ture data will be carefully analyzed In the sample and Is a very real con-^ 
slderatfon for both planning and accounting. My current bias Is that LEAs 
will have to maintain their books (census) at the program level while the 
sample data will deal .with analysis at the objective level within programs. 
The tools to analyze costs at the objective level will be provided to the 
LEAs so that they can translate their fiscal information at the program level 
Into a form such that they can analyze local real Ittes In light of state In- 
formation, or the State Department of Education can conduct an educational 
audit by selected community* 

Census data does provide a base for the management of occupational 
education by the State Department of Education throughout a current year. 
Further, the census data becomes particularly useful In the light of the In- 
formation provided by the sample In that It offers the basis for comparison 
between the sample and Individual programs within schools. A rationale for 
all census data will be mode explicit as part of the early developmental 
work of this project. 

1 1 . SAMPLE DATA 

An appropriate sample by programs over subsets of the population 
about which Inferences are to be made must be drawn. Represented In the 
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sample dbuld be a variety of meaningful student-types, within each of the 
various occupational education programs, pursuing a fairly comprehensive 
battery of program objectives over appropriate levels and within represen- 
tative school and city and town types. Such a sample will be Identified In 
a way that each local communl^ (or the State) can estimate the fit of an 
LEA Into the stratified sample data cube. (A note to myself 'Is that con- 
siderable work needs to be done on thinking through the process of reporting 
goals and objectives to allow CencourageU diversity, constrained only by 
knowledge. ) 

It Is within this sample that the detailed descriptive and ana- 
lytical data described In Monograph #1 will be attained. The sample will 
be drawn from the planning data of July (see Figure I) and corrected by the 
October I data, to detect any major d Iscrepencles. This will permit the 
gathering of Information of the sample data from the very beginning of the 
school year. The assumption here Is that everybody reading this Occasional 
Paper has read Monograph #1. The following section merely explores each 
Information type described In Monograph #1 In the light of census-sample 
■ relationships. 

III. DESCRIPTIVE DATA 

Logically, the first data to be discussed Is descriptive data. 
Product data will be discussed first, as It Is basic to all other data a'nd 
fundamental to educational management. The following Is offered as a way of 
coding descriptive data: 

D| - Input Information 
D2 - Process Information 
D3 - Product Information 
D4 - Impact Information 

8 



- Product Data 

Product data really Involves each local educational agencvV stipu- 
lating Its product goals, that Is, behavioral objectives, by program and by 
I eve I • The relationship between the census and sample Is that schools can 
compare themselves with the sample (or be compared) In terms of the specific 
objectives which they are offering. The other Information types allow addi- 
tional Individual Jzed analysis to occur; however, at this point we are deal- 
ing only with product data. 

Figure 2, below, stipulates objectives In an Imaginary, 3-year 
secondary school program. For both the census and sample, Information will 
be available which describes the program and level, by year, at whjch the 
Objectives are offered* It Is assumed that these will vary by sequence of- 
fered across programs. However, It Is anticipated the variance of terminal 
objectives within programs and across schools will be slight. Further, It 
Is assumed that each objective offered, no matter at what level of the pro- 
gram, describes a competency or capability that Is a part of the total edu- 
catlonal product as It "roils off the educational assembly line". Such a 
requirement will be stipulated In detail as the project develops. The 
sample Information system keeps track of the order or sequence In which the 
objectives are offered, as this might, and probably will, account for some of 
the end program variance, but only measures product at the end of the program. 
That would be the twelfth grade for secondary school programs, typically; the 
fourteenth grade for post-secondary school programs, or at course termination 
for adult programs or MDTA Programs. The comparative analysis for LEAs, 
therefore, Is at the point of program exit and It Is at this point that mea- 
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surement and analysts In terms of achievement data will occur. The Informa- 
tion system being developed will only concern Itself with measuring studehts 
In the sample at program completion and will provide the tools for measure- 
ment to the LEAs such that they can measure their own Students In light of 
the goals they have selected. Further, the LEAs can measure, If they wish, 
the progress of students Immediately after completion of a particular learn- 
ing experience. An Important point to be made here Is that the system being 
developed Is not an achievement monitoring process which typifies Individu- 
al Ized Instruction projects, but a large Information system Into which a 
variety of subsystems may fit. It bears repeating that the sequence In which 
the objectives are offered are more a process consideration as descrlbpd In 
Monograph #1 than product. The connect I veness between sample and census 
data Is obviously of prime Importance, as this represents a major bridge In 
the tntegrated Information system. 

D2 - Process Data 

Given Information which describes students' occupational capabili- 
ties within a sample (which is stratified over important dimensions of occu- 
pational education and Is connected to each environment In which occupational 
education Is offered) the obvious analytical question fs - "What are the 
elements of the occupational program that account for the variance or achieve- 
ment described by the product data? 11 For the purpose of developing process 
data within each sample, the following distinction is useful: (I) process 
elements, both structural and organizational, that are common to all objec- 
tives within a program such as organizational press, teachers 1 salaries, ex- 
penditure level, etc., and (2) process variables that are unique to a par- 
ticular configuration of objectives; for example,, the specific curriculum 

-8- • 



elements whfch are related to a particular set of program objectives, like 

* 

equipment, teaching strategy and Instructional time (process space of the 
IPPI model ts currently under development). Both kinds of Information are 
necessary to account for variance, and thereby allow for prediction. 

Process data should be gathered carefully during etch year so that 
analysis might be made at the end of the particular school cycle. The In- 
struments to describe process data will be made available to local educational 
agencies, so that they can determine the degree to which their programs are 
similar to those tn the sample. This provides an Important bridge or linkage 
between the sample and the population. 

There are two Interactive positions for collecting process Infor- 
mation that will be used:^ (I) that which exists; (2) the research literature 
which Indicates those elements which are most likely to have positive re- 
lationships with successful performance. The model should allow for local 
educational agencies to experiment with new processes In a way that Infor- 
mation can be attained and generalized that describes the outcomes of such 
experimentation within the model. 

D| ~ Input Data 

Like al I other data in the sample, the Input data must provide a 
bridge to the general population and at the same time detailed Information 
for careful analysis. There are two kinds of Input data other than numbers 
of students and they have been categorized as expenditure and student 
characteristic data. 

The student characteristic data will be considerably more complex 
than that which Is collected as a part of the census data (see Figure I), 
and the Instruments for detecting these differentiations will be made 

12 



available to LEAs so that they might make comparisons to the sample data-. 
The really Important criterion In terms of determining ways of looking at 
Individual differences Is. usefulness- In accounting for variance. Educa- 
tlona-l findings are typically applied to groups of people rather than to in- 
dtvldualte and, therefore, the goal Is to determine which treatments are most 
effective for which groups of students, and It Is these groups that one tries 
to Identify. For example, It would be a goal of the Information system to be 
able to say that the chances are seven Out of ten that students within this 
particular range can 'accomplish these particular goals with 'that particular 
treatment at some specified price. If we could do just that. 

It is not clear to me how expenditure information should be 
gathered, coded and stored. For example, Is It good enough for LEAs to only 
kea^ records. of cost data over a 1 1 programs, and In the sample keep track of 
cost data by programs? Can we estimate program cost foreaph LEA from over- 
all expenditure data? Should we force LEAs to (I shudder at the thought) 
keep books by program? How do we make sample population bridges? These , 
problems are for the economists to work on. In the sample, however, cost 
data must be developed for expenditure Information which costs out data not 
only by program but by objective. .We must not only know how much money was 
spent within a total program, but how much was spent on a class of students to 
demonstrate a particular capability. This is the, whole essence of product 
cost data to be discuj^sed below. I am sure that one f)as to work with clus- 
ters of objectives here. The method must be sufficiently flexible so that x 
its maintenance* In the hands of the bureaucracy doesn't tend to standardize 

programs (always an eminent danger). Cost data must also be maintained in 

« _ 

such a way that marginal as well as average costs can be determined, in addi- 
tion to requirements for cost benefit analysis, to be discussed later on. 

13 
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Resource a I location " is obviously an Important management function, and cost 
» data Is of prime importance and must be easily bridgeable to the totaf pop- 
ulation. May I repeat - my personal bias Is to keep the coSt expenditure 
data simple with the general population and detailed within the sample, 
providing tools for analysis on the part of the LEA. LEAs can then use 
these measuring tools to determine*the relationship between their reality 
and the sample and these instruments might also be used by the Department 
of Education to conduct a periodic educational audit. I recognize this as 
an extremely complex area which must be responsive to analysis needs, and 
consistent with existing practices in the State and Federal Government. 

O4 -s Impact Data 

Impact Data Is simply described as the impact of occupational edu- 
cation^ society and the student; over time, which is to say follow-up In- 
.1, formation. At this point In time, I am not at al l sure whether the impact 
data, in view of the difficulty to obtain It, should be a subsample of the 
sample or the total sample population, with double sampling for non- 
respondents. However, I suggest: there should be two samples for analysis, 
one from which general characteristics or general information is systemat- 
ically collected over time (number of jobs, job satisfaction and satls- 
f actor loess, productivity, citizenship behavior, etc.), and a very .small 
sample, perhaps # hundred, on whom a very careful analysis should be per- 
formed; on a^observat I ona I basis. There is a bunch of data you cannot get 
with a questionnaire that would be extremely useful to know. 

- 'Basically, the impact data is longitudinal, that is to say, it is 
collected on students that we know something about before they came into 
the program. We will know In some detail what the components of the educa- 
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tfonal program and the capabNltles of the students are at program exit. 
With these givens, impact data allows us to; (I) explain variance, (2) 
make predictions, (3) modify programs, (4) allocate resources, and (5) 
maintain accountability. However^ we are going to have to wait for this 
Information. In the meantime, It is going to be necessary to establish 
something of an evolving -process with impact data, such that total analysis 
can begin to occur instantly. This Is to suggest that we visualize some- 
thing of a cross-sec^ 1 ona I treatment operating concurrently in terms of ~ 
Impact data, with some fairly giant assumptions of program simi I arity, to be 
phased out as longitudinal data comes along. For example, during the first 
year, impact data will be available in October, or shortly after graduation, 
or program completion, which allows carefu I ana lysis of the Impact of pro- 
gram completors upon self and society, but there will only be limited infor- 
mation available about the process and cost of the program for those par- 
ticular products. It will take a number of years to develop the longitu- 
dinal aspects of this system and we seem to have an obligation to provide 
what Mr. Kaufman calls "good enough" data aiong the way. Obviously, these 
data must be appropriately identified, and will provide useful Information, 
brut admittedly not as good as the longitudinal impact information. 

The following point is emphasized. Descriptive data in the sample 
must be carefully gathered for analysis, and at the same time obtained in 
such a way that comparisons can be made by the state educational agency, and 
local educational agencies. The bridges between the sample and the popula- 
tion must be many and useful, and It should be fairly easy for LEA and SEA 
personnel to make comparative determinations. This Is really an essential 
ingredient to the whole process. The next section deals with analytical 
data of the sample. 
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IV. ANALYTICAL DATA 

Monograph #1 deals with six separate analytical types, although 
it'mfght be that some synthesis of these data types could occur during sys- 
tem development.. The analytical data types are listed below: 

Analytical Data « \ 
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A i - Cost Accounting Informatfon 

Cost accounting" Information has been previously described, which Is 
keeping track of the cost of occupational educational programs by program and 
program elements across school types, levels, cities and towns, etc, and by 
behavioral or product objectives. This Is obviously both essential and dif- 
ficult.-* We must know the cost of the programs, we must know the essence of 
the programs, we must know the students the programs treat, we must know the 
outcome of these programs, we must know the impact of students on society of 
these programs and all this Information must be Internally connected and 
brldgeable to the total population . All this doesn't belong here bCit It came 
to me at this moment and that way, so there It Is. 

* 

A2 - Proces^ Product Data 

1 1 • : — * * 

This also has been pretty well described. Product data Is, limited 
to those capabl 1 1 ties Which the student can demonstrate at the completion of 
the program. Process prodded informatfon assumes this Information is avail- 
able in a reliable way and that there Is a way of connecting the product data 
back through process data to Input Information. Process product information 
assumes that the information can be sliced by appropriate student character- 
istic configurations* If you will remember, the sequence of learning or of 
offering objectives by LEA Is also available Information. What process pro- 
duct Information simply does is to describe relationships between process 
and product for particular student groups. It Includes comparisons between 
student groups and between processes. It bears repeating that all Information 
types are distinguishable by specific student characteristic types. It Is my 
bias that these types be limited to a reasonable array. I think It Is absurd 
to consider Individualized Instruction as anything that will ever happen In 
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American education as It just appears to be unnecessarily Inefficient. I 
continue to subscribe to the bias that people can be Individually tracked 
into seven or twenty classes which provides a reasonably diverse structure 
for prescribing educational treatments. 

A3 - Cost Product Information 

Cost product information is really process product information with 
cost, "ground in". It simply provides a dollar estimate for process product 
Information, obviously sllceable by student characteristic configurations. 
Monograph #1 describes the usefulness of this information to educational 
managers, and It Is difficult to imagine how one could ever manage Instruc- 
tion without this data. Hopefully, this kind of information will detect 
enormous differences In the cost of preparing program completors with occu- 
pational competency, particularly in light of being able to deal with the 
data differential ly by student types. 



The next three data types deal with impact and 
assume that longitudinal impact data is available. 
As indicated above, such., information will not be avail- 
able for several years, and we therefore must manufac- 
ture crgss-sectional data which will be appropriately 
identified. 



A4 - Product Impact Information* 

If educational product is defined as the configuration by individual, 
or by classes of individuals, f of occupational competencies or capabilities at 
the completion of the educational experience, then product Impact information 
attempts to detect relationships between these behavioral patterns and their 4 * 
Impact on self and society over a period of time. Since product data is 

18 
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conn#ctable to process and Input data, the next three data types are sort of 
dominoes off this comparison* 

It might be noted that we are dealing with an Internal system. A 
separate, connectable data base that we do not have and absolutely need Is 
Information about what some describe as the validity of product data or the 
objectives that are being offered In occupational educational programs; This 
Is currently arrived at by some sort of a random, not very well organized, 
process of advisory comrnfttees. Project CAREER. (a Massachusetts development 
project) Is attempting to get some of this Information which will be coded 
Into MISOE, but such Information Is not a fundamental part of the system. 
However, product Impact data does get at this need. Over time, It will feed 
back Information Into the Internal system which describes the variance of 
different lifestyles and occupational success patterns accounted for by the 
accomplishment of various product objectives* Since this data Is all con- 
nectable, It can be differentiated by student Input type. 

Ag - Process Impact Data s 

It should be obvious by this point In time that process Impact data 
Is an attempt to detecj relationships. This Information provides the tool to 
analyze the usefulness of various processes In light of Impact over time. 

An appropriate observation might be that product Information Is the 
essential Information for the educational manager. If MISOE can reference 
product data or behavioral objectives by the events that cause them and their 
likely Impact, then they become the essentia L management tool. Again, It is 
Important to recognize that all of the product data Is coded, not only by 
Office of Education codes, but cross-referenced to DOT codes. 
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Ag - Cost Impact Data 

Cost hnpact data really Is something of a new data type which pro- J 
vldeS a tool for estimating relationships between cost and Impact. It al- 
lows a means to analyze the benefits to society and the student of occupa- 
tional education. Benefits are (by definition In Monograph #1) restricted 
to Impact variables quantifiable by the dollar and to this must be added 
other kinds of Information to get the total Impact picture. 

Also Involved In the determination or estimation of this Informa- 
tion Is the need to establish a comparable sample(s) of people who have 
recetved differential, non-occupational treatment. We leave the determina- 
tion of this to the economists, but Indicate that this Information does 
deal wtth'a larger Issue of whether to invest In occupational education or 
not, and to what extent; The objective Is to arrive at a configuration 
such that the benefits exceed the cost In light of other Investment alter- 
natives, so that one can determine what the additional benefits of addi- 
tional expenditures will be, and that these are not only known but posi- 
tive, prior to the Investment of the additional dollars. A separate sample 
must be determined by cost benefit analysis for comparison purposes* at the 
secondary level, post-secondary level, for MDTA and adult programs and any 
other Separate occupational education programs that enter into the analysis 
picture. These samples should be established at the time students begin an 
occupational program; for example, the ninth or tenth grade for secondary 
programs, the thirteenth grade for post-secondary programs, etc. 

Cone I us Ion 

Fundamentally, the purpose of this paper Is to re-examine data types 
of the Information system In light of sample population relations. Other 
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papers will deal with developmental concerns, with Occasional Paper #2 
treating developmental tasks. 

This paper references Planning Chart #1, which should be con- 
currently considered. 
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